Week 14 Covariate Effect (Forest) Plot
Application: Simulation with Parameter Uncertainty
2024-12-06
Outline
- Forest plot introduction
- Hands-on
Forest plot introduction
- Traditionally, forest plot used to display results of multiple clinical studies1.
- Extensively used in Cross-study meta-analyses.
- Point estimates with associated intervals are displayed.
- Recently, its applications in pharmacometrics were documented in the FDA popPK guidance2.
- Simulation with uncertainty on fixed effect parameters.
- Visualize the covariate effect on simulated parameters of interest (e.g., AUC, Cmax, T>MIC, etc).
A forest plot example1
![]()
Interpretation of a forest plot1
![]()
Simulation workflow
- Acquire parameter uncertainty distributions (covariance matrix, bootstrap, SIR or Bayesian posterior).
- Simulate with parameter uncertainty on fixed-effect parameters for:
- a reference subject.
- a few non-reference subjects by changing one covariate at a time (ceteris paribus).
- For each subject at each simulation replicate, PK/PD parameters of interest (e.g., AUC, Cmax, etc) were calculated.
- Standardize the PK/PD parameters of each non-reference subject relative to the reference subject.
Simulation workflow1
![]()
Advantages and disadvantages1 2
- Advantages:
- Visually appealing and intuitively understandable.
- Allow the assessment of covariate effect one at a time.
- Provide uncertainty measurements around the point estimates.
- Potential to make inferences on statistical significance and clinical relevance.
- Disadvantages:
- Do not account for correlation among covariates.
- Non-plausible scenrios can be obtained by varying covariates at a time.
- Only assessing “marginal effects”.
Forest plot with both uncertainty and variability1
![]()
Simulation workflow
- Simulate a large and realistic covariate distribution (virtual population).
- Pharmacometric simulations using the covariate distribution, between-subject variability and uncertainty.
- For each subject at each simulation replicate, PK/PD parameters of interest (e.g., AUC, Cmax, etc) were calculated.
- Stratify covariates based on quantiles, and standardize the PK/PD parameters of each non-reference subject relative to the reference subject.
Forest plot with both uncertainty and variability1
![]()
Controversy
The application of a forest plot constructed with both uncertainty and variability is not well-documented in the recent FDA popPK guidance.1
“Simulations based on uncertainty of fixed-effect parameters, BSV, and uncertainty on BSV is considered more robust and realistic, as it provides the joint effects of BSV and multiple covariates based on a database of real patients or in a virtual population with correlated covariates.”2
“Although it is technically feasible to use forest plots for visualizing between-subject variability, we strongly advise against it…blurring the use of the error bars leads to a significant risk of confusion for the viewer”3
R packages
coveffectsplot1
pmforest2
PMXforest3